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ABSTRACT 


Seventy-six  AFROTC  Cadets  studied  a  revised  version  of  the  text, 

The  Military  Justice  System,  for  four  50-minute  class  periods  distribu¬ 
ted  over  2  weeks.  Unit-mastery  tests  of  about  12  multiple -choice  items 
each  were  administered  at  11  points  throughout  the  text.  Half  of  the 
subjects  (Cadets)  received  no  knowledge  of  the  correctness  of  their 
responses  on  the  unit-mastery  test.  The  other  half  of  the  subjects 
used  chemically  treated  answer  sheets  which  immediately  indicated 
whether  or  not  the  subject’s  answer  was  correct.  A  100-item  multiple - 
choice  test  over  the  text  was  administered  to  all  subjects  2  days 
after  the  final  instruction  period.  All  subjects  had  been  informed  of 
the  final  test.  Half  of  the  subjects  in  each  of  the  above  groups  had 
been  assured  payment  of  $2.50  for  participation  in  the  study.  Each 
student  in  the  other  half  had  been  told  that  he  would  receive  $4.00 
if  he  scored  80%  or  higher  on  the  final  test,  $2.00  if  he  scored  from 
50  to  79%  and  nothing  if  he  scored  below  50%.  Compared  with  other 
subjects,  students  using  the  chemically  treated  answer  sheets  completed 
the  study  of  the  text  in  less  time  and  appeared  to  depend  on  the  mas¬ 
tery  tests  for  additional  instruction.  They  performed  significantly 
poorer  on  the  unit-mastery  tests.  On  the  final  criterion  test,  however, 
none  of  the  groups  differed  significantly.  Rather  complex  factors 
must  be  considered  in  specifying  the  optimal  conditions  of  reinforce¬ 
ment  and  incentives . 
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SECTION  I 


INTRODUCTION 


The  facilitating  effect  of  reinforcement  on  student  learning  is 
well  recognized.  Yet,  in  studies  of  applied  learning  the  term  "rein¬ 
forcement"  frequently  has  been  used  to  describe  a  variety  of  stimulus 
conditions  without  specifying  the  possible  differential  effects  re¬ 
lated  to  these  diverse  conditions.  For  example,  in  many  studies 
knowledge  of  results  for  student  responses  to  enroute  test  items  over 
the  instructional  materials  has  been  treated  as  reinforcement.  Rein¬ 
forcement  in  these  studies  is  a  condition  that  is  intrinsic  to  (i.e., 
built  into)  the  learning  materials.  In  other  studies  reinforcement 
is  a  stimulus  condition  extrinsic  to  the  learning  materials.  Here, 
the  presentation  of  reinforcement  is  contingent  upon  level  of  perfor¬ 
mance  on  the  instructional  material,  but  the  reinforcement  is  not  a 
condition  built  directly  into  the  material.  Gold  stars,  course  grades, 
and  special  awards  for  performance  serve  as  examples  of  extrinsic 
reinforcement . 

The  present  study  sought  to  investigate  the  effects  of  both  intrin¬ 
sic  and  extrinsic  reinforcement  on  student  performance  when  both  condi¬ 
tions  were  employed  in  the  same  instructional  program.  The  intrinsic 
reinforcement  condition  included  variations  in  the  knowledge  of  results 
provided  for  student  responses  to  sets  of  mastery  items  inserted  at 
various  points  in  the  instructional  material.  The  extrinsic  reinforce¬ 
ment  condition  consisted  of  variations  in  the  amount  of  money  that 
could  be  earned  by  subjects  for  acceptable  performance  on  a  final 
criterion  test  over  the  material. 


SECTION  II 
METHOD 


Subjects .  The  subjects  were  76  Air  Force  Reserve  Officers  Training 
Corps  cadets  who  were  second  semester  freshmen  enrolled  in  the  AFROTC 
program  at  Arizona  State  University.  Subjects  were  selected  at  random 
from  among  approximately  100  cadets  who  volunteered  to  participate  in 
the  experiment  during  the  time  when  they  normally  attended  AFROTC 
classes  and  drill  periods. 

Design.  The  study  employed  a  2x2  factorial  design.  The  variations  in 
intrinsic  reinforcement  conditions  served  as  one  factor  in  the  design. 
The  variations  in  extrinsic  conditions  served  as  the  second  factor. 
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Procedures .  Subjects  attended  four  50-minute  class  periods  on  a  twice- 
a-week  basis  to  read  the  instructional  material  and  answer  the  mastery 
items.  Each  subject  was  randomly  assigned  either  to  a  room  in  which 
knowledge  of  results  was  provided  for  responses  to  mastery  items  or  to 
a  room  in  which  no  knowledge  of  results  was  provided.  The  extrinsic 
reinforcement  conditions  were  randomized  within  both  the  "feedback" 
and  "no  feedback"  rooms .  Printed  instructions  explaining  the  approp¬ 
riate  extrinsic  reinforcement  contingency  and  specifying  the  time 
schedule  for  the  experiment  were  given  to  each  subject  at  the  begin¬ 
ning  of  the  first  class  meeting.  The  final  criterion  test  was  admin¬ 
istered  to  all  subjects  at  a  fifth  session  2  days  after  the  final 
instruction  period . 

Subjects  read  the  instructional  material  and  responded  to  the 
mastery  tests  at  their  own  pace.  The  "feedback"  group  received 
immediate  feedback  on  their  responses  to  each  mastery  item  as  a  func¬ 
tion  of  chemically  treated  answer  blanks .  Individuals  in  this  group 
used  special  pens  to  mark  their  responses  to  the  mastery  items .  When 
the  subject  marked  the  correct  response  blank,  the  blank  turned  red: 
when  he  marked  an  incorrect  blank,  it  turned  yellow.  Subjects  in  this 
group  were  told  that  if  their  first  response  to  an  item  was  incorrect, 
they  were  to  continue  responding  to  the  item  until  they  answered  it 
correctly.  The  "no  feedback"  group  received  no  knowledge  of  results 
on  their  responses  to  the  mastery  items . 

Two  levels  of  extrinsic  reinforcement  were  included  in  the  study. 
Instructions  for  subjects  under  the  Contingent  Reinforcement  condition 
stated  that  the  subject  would  be  paid  $4.00  if  he  scored  80%  or  higher 
on  the  final  test  over  the  instructional  material,  $2.00  if  he  scored  from 
50  to  79%,  and  nothing  if  he  scored  below  50%.  Individuals  in  the  Assured 
Reinforcement  group  stated  that  there  would  be  a  final  test  over  the 
instructional  material,  but  the  instructions  did  not  relate  the  test 
to  the  $2.50  in  any  way.  Thus,  no  extrinsic  reinforcement  based  upon 
level  of  performance  on  the  criterion  test  was  available  to  the  Assured 
Reinforcement  group. 

Materials.  The  textual  material  used  in  the  experiment  was  a  revised 
edition  of  the  Air  Force  manual,  The  Military  Justice  System  (AFROTC, 
1962).  This  text  was  original^  selected  for  revision  because  it  in¬ 
cluded  sufficient  concept  complexity  and  developmental  continuity  dir¬ 
ectly  applicable  to  Air  Force  and  ROTC  curriculum.  After  subjecting  the 
original  version  of  the  text  to  a  logical  analysis  of  objectives,  a  final 
list  of  69  specific  behavioral  objectives  was  compiled.  The  Instructional 
Specification  strategy  (Schutz,  Baker,  and  Gerlach,  1964;  Schutz,  Baker, 
Sullivan,  and  Gerlach,  1966)  was  then  employed  to  specify  the  stimulus 
conditions  required  for  the  attainment  of  these  objectives.  The  mat¬ 
erials  were  revised  on  the  basis  of  these  specified  conditions,  item 
analyses  of  the  performance  of  cadets  from  earlier  studies  on  the 
final  criterion  test,  analyses  of  interview  data  from  individual  cadets 
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who  had  read  the  materials,  and  application  of  the  gap  and  mastery 
principles  of  programed  instruction  (Silberman,  1964). 

A  set  of  131  mastery  items  was  developed  pertaining  to  the  be¬ 
havioral  objectives  specified  for  the  instructional  materials.  These 
items  were  grouped  into  eleven  unit-mastery  tests  and  inserted  at 
appropriate  points  in  the  60-page  revised  text.  Each  mastery  test 
contained  items  covering  only  the  material  from  the  section  of  the 
text  immediately  preceding  the  test.  Determination  of  the  place  in 
the  text  where  each  mastery  test  was  inserted  was  a  function  of 
optimum  length  or  logical  determination  of  appropriate  homogeneous 
blocks  of  content. 

A  final  criterion  test  of  100  three-  and  four-choice  multiple- 
choice  items  was  developed  from  an  original  pool  of  200  items .  Only 
items  with  a  difficulty  index  of  .75  or  lower  for  a  sample  of  50 
subjects  who  had  read  the  original  text  were  included  in  the  final 
100-item  test.  The  reliability  coefficient  for  the  criterion  test, 
computed  by  the  KR-20  formula  on  a  sample  of  76  subjects  who  had  read 
the  revised  text,  was  .86. 


SECTION  III 
RESULTS 


The  mean  score  for  each  group  on  the  100-item  criterion  test  is 
shown  in  Table  I.  It  is  apparent  from  the  table  that  the 
Contingent  Reinforcement  group  scored  slightly  more  than  three  points 
higher  than  the  Assured  Reinforcement  group.  However,  as  shown  in  Table 
II,  none  of  the  F-ratios  for  criterion-test  performance  was  statistically 
significant . 

Analyses  of  performance  on  the  mastery  tests,  however,  did  reveal 
important  differences  between  the  treatment  groups .  Only  3  of  the  38 
subjects  in  the  "feedback”  group  failed  to  complete  all  11  mastery  tests 
during  the  four  periods  of  the  instructional  program.  In  the  "no  feed¬ 
back"  treatment,  however,  15  of  the  38  individuals  completed  only  10  or 
fewer  tests  and  failed  to  reach  the  final  mastery  test.  Clearly,  the 
"no  feedback"  group  was  spending  more  time  studying  either  the  textual 
material  or  the  mastery  items  than  was  the  "feedback"  group. 

Descriptive  statistics  relating  to  the  mean  standard  score  on  unit- 
mastery  tests  completed  by  each  subject  are  shown  in  Table  III.  Since 
data  on  mastery-test  scores  were  not  available  on  4  subjects  from  the 
contingent-reinforcement-plus-feedback  cell  when  the  statistical 
analyses  were  performed,  these  subjects  were  assigned  the  computed  mean 
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standard  score  for  their  cell  on  mastery-test  performance.  This  accounts 
for  the  slight  variation  of  the  grand  mean  score  (49.85)  from  a  grand 
mean  of  50.00.  .  The  results  of  the  analysis  of  variance  on  these  data 
are  presented  in  Table  IV. 

The  data  in  Tables  III  and  IV  reveal  that  the  ”no  feedback”  sub¬ 
jects  performed  considerably  better  on  the  mastery  items  than  did  the 
”feedback”subjects .  The  mean  standard  score  for  mastery-test  perfor¬ 
mance  is  5.52  standard  score  points  higher  for  the  "no  feedback"  group. 
This  effect  is  significant  at  the  .001  level  of  significance.  Neither 
the  extrinsic  reinforcement  contingency  effect  nor  the  interaction 
between  intrinsic  and  extrinsic  reinforcement  approached  statistical 
significance. 


Table  I 

CRITERION  TEST  MEAN  SCORES 


Extrinsic 

Reinforcement 

Intrinsic  Reinforcement 

Totals 

Feedback 

No 

Feedback 

N 

Score 

N 

Score 

N 

Score 

Contingent 
($4  -  $2  -0) 

19 

61.37 

19 

61.32 

38 

61.35 

Assured 

($2.50) 

19 

57.00 

19 

59.63 

38 

58.32 

Totals 

38 

59.18 

38 

60.47 

76 

59.83 

Table  II 

ANALYSIS  OF  VARIANCE  FOR  CRITERION  TEST  SCORES 


Source 

df 

SS 

MS 

F 

Extrinsic 

1 

174 

174 

2.69 

Intrinsic 

1 

32 

32 

.50 

E  x  I 

1 

34 

34 

.53 

Within 

72 

4653 

64.6 

Totals 

75 

4893 

4 


Table  III 


MASTERY  TEST -MEAN  STANDARD  SCORES 


Table  IV 


ANALYSIS 

OF  VARIANCE 

FOR  MASTERY 

TEST  PERFORMANCE 

Source 

df 

SS 

MS 

F 

Extrinsic 

1 

8.08 

8.08 

.31 

Intrinsic 

1 

577.94 

577.94 

22.08* 

E  x  I 

1 

7.40 

7.40 

.28 

Within 

72 

1884.07 

26.17 

Totals 

75 

2477.49 

*  p  <.001 
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SECTION  IV 


DISCUSSION 


The  results  of  the  study  suggest  that  there  were  important  diff¬ 
erences  between  the  ’'feedback”  and  "no  feedback"  treatment  groups 
in  the  strategies  that  they  employed  to  learn  the  instructional  mater¬ 
ial.  The  better  performance  of  the  "no  feedback"  group  on  the  mastery 
tests  and  the  failure  of  15  subjects  from  this  group  to  finish  the 
instructional  program  indicate  that  the  "no  feedback"  subjects  ex¬ 
pended  more  time  and  effort  attempting  to  learn  from  the  prose  textual 
material.  For  these  individuals,  of  course,  this  is  the  only  instruc¬ 
tional  material  in  the  text.  The  "f eedback" subjects ,  on  the  other 
hand,  apparently  neglected  the  textual  material  to  some  degree  and 
used  the  instructional  value  of  the  immediate  feedback  to  their  mastery- 
item  responses .  Such  a  procedure  would  account  for  their  greater  speed 
in  working  through  the  textual  material  and  their  inferior  performance 
on  the  mastery  items.  That  the  "feedback"  subjects  were  successful  in 
learning  from  the  immediate  feedback  to  their  mastery-test  responses 
is  demonstrated  by  their  subsequent  performance  on  the  criterion  test. 
This  criterion-test  performance  was  comparable  to  that  of  the  "no 
feedback"  group,  even  though  their  mastery-test  performance  was  signifi¬ 
cantly  inferior. 

An  interesting  phenomenon  to  note  here  is  the  apparent  sensitivity 
of  the  learner  to  subtle  procedural  cues  implicit  in  the  instructional 
material.  For  example,  one  might  predict  that  both  the  "feedback"  and 
"no  feedback"  groups  would  study  equally  hard  on  the  textual  material 
and  that  no  difference  would  occur  in  mastery-test  performance  between 
the  two  groups .  Since  feedback  is  not  received  until  after  the  learner 
responds  to  a  mastery  item,  one  would  expect  that  on  subsequent  items 
over  the  same  material  (e.g.,  the  criterion  test)  the  feedback  would 
result  in  an  advantage  for  the  individuals  receiving  it.  However,  it 
appears  that  "feedback"  subjects  quickly  observe  that  they  need  not 
labor  over  the  textual  materials  to  learn  the  material  to  be  covered 
on  the  criterion  test.  Where  individuals  in  the  "no  feedback"  group 
may  choose  to  look  on  the  preceding  pages  for  the  correct  answer  to 
a  puzzling  mastery  item,  a  subject  in  the  "feedback"  group  can  employ 
the  easier  and  simpler  expedient  of  marking  in  succession  his  highest- 
order  response  choice  until  the  feedback  indicates  a  correct  response. 

How  can  one  capitalize  upon  the  advantages  of  the  intrinsic  rein¬ 
forcement  involved  in  the  immediate  feedback  procedure  on  the  mastery 
items  while  at  the  same  time  maintaining  the  control  of  the  textual 
material  over  the  reader's  learning?  One  possible  procedure  would  be 
to  provide  the  learner  with  extrinsic  reinforcement  for  acceptable 
performance  on  the  mastery  items,  as  well  as  for  acceptable  perfor¬ 
mance  on  the  criterion  test .  Thus ,  performance  of  the  "feedback" 
subjects  on  the  mastery  tests  should  be  improved  because  of  the  extrinsic 
reinforcement  associated  with  good  mastery  performance.  The  immediate 
feedback  on  these  items  should  still  serve  to  facilitate  subsequent  per¬ 
formance  on  the  criterion  test. 
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A  final  word  should  be  said  concerning  the  effect  of  extrinsic 
reinforcement  in  using  instructional  materials.  The  differences 
in  the  levels  of  the  monetary  reinforcement  contingency  employed  in  the 
present  study  were  not  of  sufficient  strength  to  significantly  affect 
student  performance.  However,  there  is  little  doubt  that  extrinsic 
reinforcement  is  required  in  the  learning  task  to  maintain  control  of  the 
instructional  material  over  student  responses.  In  the  military  setting 
such  reinforcers  as  passes,  coffee  or  cigarette  breaks,  and  prefer¬ 
ential  assignments  may  be  employed  to  develop  and  maintain  desired  learner 
responses  to  learning  materials.  The  use  of  these  and  other  available 
reinforcers  by  the  military  instructor  is  an  essential  technique  for 
maximizing  the  effectiveness  of  instructional  materials.  No  matter  how 
excellent  the  quality  of  the  material,  the  student  will  not  learn  it 
well  unless  he  is  provided  with  an  incentive  for  doing  so. 
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